Estimation and Variational Methods for Gradient Algorithm Generation

Natural Conjugate Gradient in Variational Inference

Variational methods for approximate inference in machine learning often adapt a parametric probability distribution to optimize a given objective function. This view is especially useful when applying variational Bayes (VB) to models outside the conjugate-exponential family. For them, variational Bayesian expectation maximization (VB EM) algorithms are not easily available, and gradient-based m...

A Relaxed Extra Gradient Approximation Method of Two Inverse-Strongly Monotone Mappings for a General System of Variational Inequalities, Fixed Point and Equilibrium Problems

Noisy Natural Gradient as Variational Inference

Variational Bayesian neural nets combine the flexibility of deep learning with Bayesian uncertainty estimation. Unfortunately, there is a tradeoff between cheap but simple variational families (e.g. fully factorized) or expensive and complicated inference procedures. We show that natural gradient ascent with adaptive weight noise implicitly fits a variational posterior to maximize the evidence ...

Blind Deconvolution with Non-local Sparsity Reweighting

Blind deconvolution has made significant progress in the past decade. Most successful algorithms are classified either as Variational or Maximum a-Posteriori (MAP ). In spite of the superior theoretical justification of variational techniques, carefully constructed MAP algorithms have proven equally effective in practice. In this paper, we show that all successful MAP and variational algorithms...

A Novel DOA Estimation Approach for Unknown Coherent Source Groups with Coherent Signals

In this paper, a new combination of Minimum Description Length (MDL) or Eigenvalue Gradient Method (EGM), Joint Approximate Diagonalization of Eigenmatrices (JADE) and Modified Forward-Backward Linear Prediction (MFBLP) algorithms is proposed which determines the number of non-coherent source groups and estimates the Direction Of Arrivals (DOAs) of coherent signals in each group. First, the MDL...